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DETAILED ACTION 
Response to Amendment 

The amendment filed 03/09/2006 has been entered and made of record. Claims 
163-198 are pending. Claims 1-162 have been cancelled. 



Response to Arguments 

Applicant's arguments have been considered but are moot in view of the new 
ground(s) of rejection. Additionally, in regards to Pryor (2002/0036617 A1), Applicant 
argues Pryor fails to teach a method of for manipulating a virtual object with a virtual 
interface component. However, the method/system of Pryor allows the user to 
manipulate virtual objects via an input device that is mapped accordingly so as on 
screen, the virtual object is being manipulated. Furthermore, the input device of Pryor 
does not need to be represented within the graphical user interface as the physical 
features of the device but rather, can take an entirely different appearance depending 
on the user's preference, thus being a virtual input device. Pryor discloses using retro- 
reflective material [01 1 1] or targets [01 13] adhered to input devices, or also using the 
users own hands as input devices. The program of computer (630) recognizes this 
motion of fingernails (635) and (636) seen by cameras (640) and (641) connected to the 
computer which processes their image according to the manipulation [0334]. 
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Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

1. Claims 163, 164, 170, 171, 173, 174, 177, 178, 183, 188, 189, 194-196 are 
rejected under 35 U.S.C. 102(b) as being anticipated by Pryor (2002/0036617 A1). 

Pryor teaches a method and apparatus for inputting position, attitude (orientation) 
or other object characteristic data to computers for the purpose of Computer Aided 
learning, Teaching, Gaming, Toys, Simulations, Aids to the disabled, Word Processing 
and other applications. 

In regards to claims 163, 173, 174, 177, 188, 195, 196 - 

Fig. 1a shows the basic invention of Pryor. A user (5), desires to point at an object 
(6) represented electronically on the screen (7) and cause the pointing action to register 
in the software contained in computer (8) with respect to that object (a virtual object), in 
order to cause a signal to be generated to the display (7) to cause the object to activate 
or allow it to be moved (e.g., with a subsequent finger motion or otherwise) (said 
generating one or more interface devices to alter and generate one or more two- 
dimensional or three-dimensional virtual objects) [0109]. This system can provide 
highly accurate position and orientation information in up to 6 degrees of freedom (said 
wherein said devices can control N degrees of freedom of said virtual objects, 
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claim 161 : wherein N degrees of freedom is 6 degrees of freedom) [0134]. 
Furthermore, the invention of Pryor can "alias" the interface devices, such as represent 
the input device, such as a pen, as a paintbrush on the screen (7) (said providing one 
or more three-dimensional virtual tools to a user for said spatial manipulation or 
said two-dimensional or said three-dimensional entertainment) [0275]. Fig. 9 
illustrates a means for aiding the movement of a person's hands while using the 
invention of Pryor in multiple degrees of freedom movement. Shown in Fig. 9a, joystick 
(905) has ball (910) attached to it's end in which the data from datums on the ball 
position is taken optically by a video camera in up to 6-axes [0382]. Two joysticks may 
be used (said generating one or more interface devices; associating said interface 
devices in conjunction with each other to alter one or more two-dimensional or 
three-dimensional virtual components) [0383]. 

In regards to claims 164, 178, 189 - 

The datums on an object can be known a priori relative to other points on the object, 
and to other datums, by providing it to a user via a CD ROM disc or other computer 
interface storage medium having this data [0143] (said virtual components are a 
software representation of physical input devices). 

In regards to claims 170, 183, 194- 

Referring to Fig. 3, Pryor uses the example of an artist with tools, such as an eraser 
(said 3D virtual tool is an eraser tool, wherein said tool is used to remove a region 
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of a virtual surface) [0274]. The artist would pick up the tool and normally use the tool 
over the surface of a sheet of paper, display screen or projection of computer display. 
The application software would not only trace the path of the tip of the targeted work 
too, but also treat the tool as though it were the object, i.e. eraser. 

In regards to claim 171 - 

Fig. 6 illustrates parts of the user, such as the hands can describe motion or 
position signatures and sequences of considerable utility [0330]. Some natural actions 
of this type are grip, pinch, grasp (said grab), stretch, bend, twist, rotate, screw, point, 
hammer, throw (said move), and other natural actions learned in life [0331]. The 
method and system of Pryor detects the position of one finger, two fingers of one hand, 
one finger of each hand, two hands (said using two or more of said physical input 
devices coincidentally), or relative motion/position of any of the above with respect to 
the human or the computer camera system or the screen [0333]. These actions can 
cause objects depicted on the screen to be acted on [0334]. Thus, two hands can be 
used where one hand performs one action, such as grabbing, while the second hand 
performs a second action, such as lengthening. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 167-169, 172, 175, 176, 180-182, 184-186, 191-193, 197, and 198 are 

rejected under 35 U.S.C. 103(a) as being unpatentable over Pryor (US Publication No. 
US 2002/0036617 A1). 

In regards to claims 167, 168, 180, 181, 191, 192- 

Pryor teaches the limitations of claims 167, 168, 180, 181, 191, 192 with the 
exception of disclosing an input device to cut a virtual object. However, Fig. 6 illustrates 
parts of the user, such as the hands can describe motion or position signatures and 
sequences of considerable utility [0330]. Some natural actions of this type are grip, 
pinch, grasp, stretch, bend (said bend virtual objects), twist, rotate, screw, point, 
hammer, throw, and other natural actions learned in life [0331]. The method and 
system of Pryor detects the position of one finger, two fingers of one hand, one finger of 
each hand, two hands (said using two or more of said physical input devices 
coincidentally), or relative motion/position of any of the above with respect to the 
human or the computer camera system or the screen [0333]. These actions can cause 
objects depicted on the screen to be acted on [0334]. Thus, two hands can be used 
where one hand performs one action, such as grabbing, while the second hand 
performs a second action, such as lengthening. 

Additionally, as shown in Fig. 15, a sword video game can incorporate the method 
of Pryor. It would have been obvious to one skilled in the art that the sword (1502) held 
by player (1501) can virtually cut the "bad guys" within the game [0441]-[0442]. 
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Furthermore, Pryor teaches that the invention allows the user to "alias" any object 
with any other object [0281]. Thus, the hands of the user can be aliased as a sword 
that can cut the virtual objects (said cut virtual objects). 

In regards to claims 169, 182, 193 - 

Pryor teaches the limitations of claims 169, 182, 193 with the exception of 
disclosing the grabbing tools, pointing tools, and gripping tools resembling specific 
physical forms. However, Pryor discloses using retro-reflective material [0111] or 
targets [01 13] adhered to input devices. 

With the aid of Fig. 6, Pryor teaches the act of grabbing where the thumb (601) 
and the first finger (602) are near an object such as a 3D graphic rendition of a cow 
(610) displayed on the screen (615). As the fingers are converged in a pinching motion 
depicted as dotted lines (620), the program of computer (630) recognizes this motion of 
fingernails (635) and (636) seen by cameras (640) and (641) connected to the computer 
which processes their image, as a pinch/grasp motion and can either cause the image 
of the cow to be compressed graphically [0334]. Although Pryor is silent about other 
grabbing tools, Pryor discloses using retro-reflective material [0111] or targets [0113] 
adhered to input devices. Therefore, such material can be adhered to kitchen tongs, 
pincers, scissors, and or tweezers that mimic the same pinching motion as the thumb 
(601) and first finger (602). 

Parts of the user, such as the hands can describe motion or position signatures 
and sequences of considerable utility, such as pointing [0331]. This action can cause 
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objects depicted on a screen to be acted on, by sensing [0334]. Thus, similar to the 
grabbing tools, the retro-reflective material [0111] or targets [0113] can be adhered to 
input devices, such as a flash light, spray-paint can or camera to mimic the same 
pointing motion as the user's finger (said spray can). 

These tools can be considered targeted tools, where a targeted work tool can be 
a toy model of the real world tool or the tool itself, helping the user immediately visualize 
the properties of the tool in the computer program [0236]. Thus, it would have been 
obvious to use such physical items in order to provide a realistic simulation of the users 
motion in regards to the graphic rendition on the screen (615). 

In regards to claims 172, 175, 176, 197, 198 - 

Pryor teaches the limitations of claim 172, 175, 176, 197, 198 with the exception of 
modifying an axis of rotation. However, Fig. 6 illustrates parts of the user, such as the 
hands can describe motion or position signatures and sequences of considerable utility 
[0330]. Some natural actions of this type are grip, pinch, grasp, stretch, bend, twist, 
rotate (said rotate said virtual object), screw, point, hammer, throw, and other natural 
actions learned in life (said using tools to move virtual component to a desired 
location) [0331]. The method and system of Pryor detects the position of one finger, 
two fingers of one hand, one finger of each hand, two hands, or relative motion/position 
of any of the above with respect to the human or the computer camera system or the 
screen [0333]. These actions can cause objects depicted on the screen to be acted on 
[0334]. Thus, two hands can be used where one hand performs one action, such as 
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grabbing (said grabbing with said first grabbing tool), while the second hand 
performs a second action, such as rotating (said rotating with grabbing tool). 
Although Pryor is silent about an axis of rotation, and axis of rotation would be needed 
for the user to rotate the virtual object. Furthermore, the invention of Pryor permits the 
virtual object to be moved or altered in position. Thus, when the virtual object's position 
has been altered, either rotated a certain degree, etc., the axis of rotation as then been 
modified (said modify an axis of rotation). 

In regards to claims 184-186 - 

Pryor teaches the limitations of claim 184-186 with the exception of a closed curve. 
However, Fig. 1a shows the basic invention of Pryor. A user (5), desires to point at an 
object (6) represented electronically on the screen (7) and cause the pointing action to 
register in the software contained in computer (8) with respect to that object (a virtual 
object), in order to cause a signal to be generated to the display (7) to cause the object 
to activate or allow it to be moved (e.g., with a subsequent finger motion or otherwise) 
(said generating one or more interface devices to alter and generate one or more 
two-dimensional or three-dimensional virtual objects) [01 09]. This system can 
provide highly accurate position and orientation information in up to 6 degrees of 
freedom [0134]. Furthermore, the invention of Pryor can "alias" the interface devices, 
such as represent the input device, such as a pen, as a paintbrush on the screen (7) 
(said providing one or more three-dimensional virtual tools to a user for said 
spatial manipulation or said two-dimensional or said three-dimensional 
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entertainment) [0275]. Fig. 9 illustrates a means for aiding the movement of a person's 
hands while using the invention of Pryor in multiple degrees of freedom movement 

Fig. 6 illustrates parts of the user, such as the hands can describe motion or 
position signatures and sequences of considerable utility [0330]. Some natural actions 
of this type are grip, pinch, grasp, stretch, bend (said bend), twist, rotate (said rotate), 
screw, point, hammer, throw, and other natural actions learned in life [0331]. Although 
Pryor is silent about other grabbing tools, Pryor discloses using retro-reflective material 
[01 11] or targets [0113] adhered to input devices. Therefore, such material can be 
adhered to kitchen tongs, pincers, scissors, and or tweezers that mimic the same 
pinching motion as the thumb (601) and first finger (602). 

Parts of the user, such as the hands can describe motion or position signatures 
and sequences of considerable utility, such as pointing (said point) [0331]. This action 
can cause objects depicted on a screen to be acted on, by sensing [0334]. Thus, 
similar to the grabbing tools, the retro-reflective material [01 1 1] or targets [01 13] can be 
adhered to input devices, such as a flash light, spray-paint can or camera to mimic the 
same pointing motion as the user's finger. 

These tools can be considered targeted tools, where a targeted work tool can be 
a toy model of the real world tool or the tool itself, helping the user immediately visualize 
the properties of the tool in the computer program [0236]. Thus, it would have been 
obvious to use such physical items in order to provide a realistic simulation of the users 
motion in regards to the graphic rendition on the screen (615). 
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Although Pryor does not explicitly teach a closed curve, it is would have been 
obvious to one of ordinary skill in the art that the method and system of Pryor would 
allow such a motion to occur since the method and system of Pryor mimics some 
natural actions such as bending and other natural actions learned in life [0331]. 
Therefore, with such a bending motion, the virtual object can be bent to a closed curve. 
Furthermore, although not explicitly taught, it would have been implicit that if the curve 
is bent closed, a volume is generated. 

In regards to claim 187 - 

Pryor teaches the limitations of claim 187 with the exception of molecule 
components. However, Fig. 1a shows the basic invention of Pryor. A user (5), desires 
to point at an object (6) represented electronically on the screen (7) and cause the 
pointing action to register in the software contained in computer (8) with respect to that 
object (a virtual object), in order to cause a signal to be generated to the display (7) to 
cause the object to activate or allow it to be moved (e.g., with a subsequent finger 
motion or otherwise) (said generating one or more interface devices to alter and 
generate one or more two-dimensional or three-dimensional virtual objects) 
[0109]. This system can provide highly accurate position and orientation information in 
up to 6 degrees of freedom [0134]. Furthermore, the invention of Pryor can "alias" the 
interface devices, such as represent the input device, such as a pen, as a paintbrush on 
the screen (7) (said providing one or more three-dimensional virtual tools to a user 
for said spatial manipulation or said two-dimensional or said three-dimensional 
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entertainment) [0275]. Fig. 9 illustrates a means for aiding the movement of a person's 
hands while using the invention of Pryor in multiple degrees of freedom movement 
Fig. 6 illustrates parts of the user, such as the hands can describe motion or 
position signatures and sequences of considerable utility [0330]. Some natural actions 
of this type are grip, pinch, grasp, stretch, bend (said bend), twist, rotate, screw, point, 
hammer, throw, and other natural actions learned in life [0331]. Although Pryor is silent 
about other grabbing tools, Pryor discloses using retro-reflective material [01 1 1] or 
targets [0113] adhered to input devices. Therefore, such material can be adhered to 
kitchen tongs, pincers, scissors, and or tweezers that mimic the same pinching motion 
as the thumb (601) and first finger (602). 

Parts of the user, such as the hands can describe motion or position signatures and 
sequences of considerable utility, such as pointing (said point) [0331]. This action can 
cause objects depicted on a screen to be acted on, by sensing [0334]. Thus, similar to 
the grabbing tools, the retro-reflective material [0111] or targets [01 1 3] can be adhered 
to input devices, such as a flash light, spray-paint can or camera to mimic the same 
pointing motion as the user's finger. Additionally, as shown in Fig. 15, a sword video 
game can incorporate the method of Pryor. It would have been obvious to one skilled in 
the art that the sword (1502) held by player (1501) can perform the act of cutting [0441]- 
[0442]. Furthermore, Pryor teaches that the invention allows the user to "alias" any 
object with any other object [0281]. Thus, the hands of the user can be aliased as a 
sword that can cut the virtual objects (said cut virtual objects). 
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These tools can be considered targeted tools, where a targeted work tool can be 
a toy model of the real world tool or the tool itself, helping the user immediately visualize 
the properties of the tool in the computer program [0236]. Thus, it would have been 
obvious to use such physical items in order to provide a realistic simulation of the users 
motion in regards to the graphic rendition on the screen (615). 

Although Pryor does not explicitly disclose molecule components, the virtual 
objects of Pryor are manipulated where the user can cut (i.e. break bonds), move, rotate 
and draw. The molecule components of the current application are used as virtual 
objects within the graphical user interface, and thus regardless of the appearance of the 
virtual objects, they are being manipulated in the same fashion as the virtual objects of 
Pryor. 

3. Claim 165 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pryor 
(US Publication No. US 2002/0036617 A1) in view of Harvill et al. (US Patent No. 
6,222,523 B1). 

Pryor teaches the limitation of claim 165 with the exception of teaching the virtual 
component coinciding with the tool's physical form. However, Harvill et al. attaching a 
tactile stimulus to the users body when a virtual operator, created by movements of the 
physical operator, encounters a virtual object defined by a computer. 

Referring to Fig. 6 of Pryor, Fig. 6 illustrates other movements such as gripping or 
touching. Parts of the user such as the hands, can describe motion or position 
signatures and sequence of considerable utility, e.g., grip, pinch, grasp, stretch, bend, 
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twist, rotate, screw, point, hammer, throw [0331]. These actions can cause objects 
depicted on a screen to be acted on, by sensing the users hand relative to the human or 
the computer camera system or the screen [0333]. 

Fig. 1 of Harvill et al. is an overall perspective view of a data processing system (4) 
wherein movements of a part of a physical body (10) of an operator (12) are converted 
into a virtual operator (16) (said virtual form of said tool is an iconic virtual 
component) for manipulating a virtual object (20) represented within data processing 
system (4) [col. 2, lines 65 - 68; col. 3 lines 1 - 4]. As seen, the virtual operator (16) 
takes the form of the input device, i.e. the hand of the operator (10). As seen, the virtual 
form takes the shape of a 2D object. The movements of a hand (22) of physical 
operator (12) are converted into a virtual operator (16) through a glove (24). Thus, as 
the hand (22) moves, the virtual operator (16) mimics the movements on the display 
(said altering the relationship between said tool and its corresponding 3D virtual 
component mapping said tool to said corresponding virtual component). It would 
have been obvious to one of ordinary skill in the art to have the virtual form also take a 
3D shape since the data processing system (4) converts the movements of a physical 
body (10) of an operator (12) and the movements are within a 3D space. As shown in 
Fig. 2 of Harvill et al., a programmable interval timer (28) receives data from a CPU (32) 
of data processing system (4) through a bus (36). Internal timer 928) provides a signal 
to a stimulus circuit (40) through a bus (44) whenever virtual operator (16) encounters 
virtual object (20) [col. 3, lines 15 - 22]. Although Harvill et al. is silent about generating 
an iconic form when the virtual component is close enough to react with the virtual 
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objects, the virtual operator (16) reacts with the virtual object (20), thus altering it's state 
when near the virtual object (20). 

Therefore, it would have been obvious to one of ordinary skill in the art to include 
the conversion method of Harvill et al. to display a virtual operator coinciding with the 
physical form of the input device of Pryor so the user can visually see the relationship 
between the input device and the virtual object on the screen, providing an easier 
means of manipulations of the virtual object on the screen. 

4. Claims 166, 179, and 190 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Pryor (2002/0036617 A1) in view of Kumar et al (6,222,465 B1). 

Pryor teaches the limitations of claims 166, 179, and 190 with the exception of 
disclosing smoothing or texturizing a portion of the virtual object. However Kumar 
teaches applying textures to the virtual objects. 

When the user selects and executes the CHANGE CHARACTERISTIC 
command, the user can alter the colors, shininess types, and textures via the 
PALETTE (304) [col. 7, lines 15-41]. 

It would have been obvious to one of ordinary skill in the art to apply texturing as 
taught by Kumar to the virtual objects of Pryor in order to provide a more real world, or 
life like qualities to the user within the virtual environment. 
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Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michelle K. Lay whose telephone number is (571) 272- 
7661. The examiner can normally be reached on Monday through Thursday from 
7:30am to 5:00pm. The examiner can also be reached on alternate Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee M. Tung, can be reached at (571) 272-7794. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
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published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Michelle K. Lay 
Patent Examiner 
Division 2628 
05.18.2006 mkl 
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